Rejoinder: One-step Sparse Estimates in Nonconcave Penalized Likelihood Models By

Most traditional variable selection criteria, such as the AIC and the BIC, are (or are asymptotically equivalent to) the penalized likelihood with the L0 penalty, namely, pλ(|β|) = 2λI (|β| = 0), and with appropriate values of λ (Fan and Li [7]). In general, the optimization of the L0-penalized likelihood function via exhaustive search over all subset models is an NP-hard computational problem. Donoho and Huo [3] and Donoho and Elad [2] show that, under some conditions, the solution to the L0-penalized problem can be found by solving a convex optimization problem of minimizing the L1-norm of the coefficients, when the solution is sufficiently sparse. In other words, the NP-hard best subset variable selection can be solved by efficient convex optimization algorithms under the sparsity assumption. This sheds light on variable selection for high-dimensional models, and motivates us to use continuous penalties, such as the L1 penalty, rather than the discontinuous penalties, including the L0 penalty. The penalized likelihood procedure with the L1 penalty coincides with LASSO. In the same spirit of LASSO, the penalized likelihood with a nonconcave penalty, such as the SCAD penalty, has been proposed for variable selection in Fan and Li [5]. LASSO and the SCAD represent the two main streams of penalization method for variable selection in the recent literature. Although both methods operate as continuous thresholding rules, they appear to be very different theoretically and computationally. The SCAD is asymptotically unbiased and enjoys the oracle properties which LASSO cannot possess (Zou [16]). On the other hand, LASSO uses the L1 penalty as opposed to the concave SCAD penalty, thus being computationally more friendly than the SCAD. Computational efficiency of a statistical method determines its popularity to a great extent. Interestingly, this argument is evidenced by the evolution of LASSO.